Systems and methods for displaying guidance data based on updated deformable imaging data

ABSTRACT

Presented herein are methods, systems, and computer-readable medium for presenting imaging data related to an anatomical site. These include obtaining a first set of imaging data related to the anatomical site and tracking units at the anatomical site and, thereafter, optionally, obtaining a second set of imaging data related to the anatomical site. A deformed version of the first set of imaging data is then determined based on the relative arrangements of one or more of the tracking units at the time when the first set of imaging data is obtained and when the second set of imaging data is obtained. Then the relative emplacements of the second set of imaging data set and of the deformed version of the first set of imaging data set are determined and used, along with the second set of imaging data set and the deformed version of the first set of imaging data, as a basis for displaying image guidance data.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.61/068,469, filed Mar. 7, 2008.

BACKGROUND

Surgeons often need to be able to look at both pre-operative data, suchas computed tomography (“CT”) scans and magnetic resonance imaging(“MRI”) scans, as well as intra operative data, such as two dimensional(“2D”) ultrasound or three dimensional (“3D”) ultrasound while they arein the operating room. Normally, doctors view the CT scans andultrasound on separate displays and must use their imaginations in orderto correlate the information in the two images. This is a difficultspatial task for the surgeons to accomplish. Further, when a targetanatomical site is located within soft tissue, the pre-operative data isout of date with respect its pre-operative form because of the movement,compression and reorientation of the soft tissue and, therefore, it isdifficult or impossible for the surgeon to appropriately utilize thepre-operative data during the operation.

Previous systems have attempted to aid the surgeon using computer visionregistration techniques. Example systems are described in, among otherpapers, Aylward et al., Analysis of the Parameter Space of a Metric forRegistering 3D Vascular Images, in W. Niessen and M. Viergever (Eds.),MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION—MICCAI 2001,pp. 932-939; and Aylward et al, Intra-Operative 3D UltrasoundAugmentation, Proceedings of the IEEE International Symposium onBiomedical Imaging, Washington, D.C., July 2002. The problem with thesesystems however is the massive computational strain required by theregistration techniques.

SUMMARY

Presented herein are methods, systems, and computer-readable medium forpresenting imaging data related to an anatomical site. These includeobtaining a first set of imaging data related to the anatomical site andtracking units at the anatomical site and, thereafter, optionally,obtaining a second set of imaging data related to the anatomical site. Adeformed version of the first set of imaging data is then determinedbased on the relative arrangements of one or more of the tracking unitsat the time when the first set of imaging data is obtained and when thesecond set of imaging data is obtained. Then the relative emplacementsof the second set of imaging data and the deformed version of the firstset of imaging data are determined and used, along with the second setof imaging data and the deformed version of the first set of imagingdata, as a basis for displaying image guidance data.

Presented herein are methods, systems, and computer-readable medium forpresenting imaging data related to an anatomical site, that includeobtaining, at a first time, a first set of imaging data related to theanatomical site. Thereafter, tracking information for a movable imagingdevice controlled by a user is obtained at a second time, after thefirst time. Then desired emplacement information is determined for animage of the first set of imaging data based on the trackinginformation. Finally, image guidance data is determined for displaybased on the first set of imaging data and the desired emplacementinformation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts one embodiment of a system capable of updating deformableimaging data.

FIG. 2 illustrates an example of an anatomical site with tracking units.

FIG. 3 illustrates the deformation of an anatomical site with trackingunits.

FIG. 4 depicts an embodiment of a tracking unit.

FIG. 5 illustrates another example of an anatomical site withindeformable tissue with tracking units.

FIG. 6 depicts an example process for providing guidance data based onupdated deformable tracking information.

FIGS. 7A-7D depict marking and viewing features in imaging data.

FIG. 8 depicts an embodiment of image guidance data in which first setof imaging data is presented with second set of imaging data.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

I. Overview

FIG. 1 depicts merely one exemplary embodiment of a system 100 capableof updating deformable imaging data. There are numerous other possibleembodiments of system 100, for example, numerous of the depicted modulesmay be joined together to form a single module and may even beimplemented in a single computer or machine. Further, the positionsensing units 110 and 140 may be combined and track all relevanttracking units 145 and movable imaging units 155, as discussed in moredetail below. Additionally, imaging unit 150 may be excluded and onlyimaging data from the image guidance unit 130 may be shown on displayunit 120. These and other possible embodiments are discussed in moredetail below. Numerous other embodiments will be apparent to thoseskilled in the art and are covered by the invention as claimed.

In the pictured embodiment, the system 100 comprises a first positionsensing unit 110, a display unit 120, and the second position sensingunit 140 all coupled to an image guidance unit 130. In some embodiments,the first position sensing unit 110, the displaying unit 120, the secondposition sensing unit 140, and the image guidance unit 130 are allphysically connected to stand 170. The image guidance unit 130 may beused to produce images 125 that are presented on display unit 120. Asdiscussed more below, the images 125 shown on the display unit 120 bythe image guidance unit 130 may be determined based on imaging data,such as a CT scan, MRI, open-magnet MRI, optical coherence tomography,positron emission tomography (“PET”) scans, fluoroscopy, ultrasound, orother preoperative or intraoperative anatomical imaging data and any 3Danatomical imaging data. The images 125 produced may also be based onintraoperative or real-time data obtained using a movable imaging unit155, which is coupled to imaging unit 150. Real-time may implyinstantaneous or near-instantaneous obtaining of data. Real-time mayalso imply that it is taken with the intention to be used immediately.Imaging unit 150 may be coupled to image guidance unit 130. In someembodiments, imaging unit 150 may be coupled to a second display unit151. The second display unit 151 may present imaging data from imagingunit 150. The imaging data displayed on display unit 120 and displayedon second display unit 151 are not necessarily the same. In someembodiments, the imaging unit 150 is an ultrasound machine 150, themovable imaging device 155 is an ultrasound transducer 155 or ultrasoundprobe 155, and the second display unit 151 is a display associated withthe ultrasound machine 150 that shows the imaging data from theultrasound machine.

The second position sensing unit 140 is coupled to one or more trackingunits 145. The second position sensing unit 140 and tracking units 145may together comprise a magnetic tracking system, an optical trackingsystem, or any other appropriate tracking system. The second positionsensing unit 140 and tracking units 145 may be used to track thedeformation of tissue at a target anatomical site on user 160. User 160may be in an operating room, lying on an operating table, such asoperating table 180, or in any other appropriate place or position. Invarious embodiments, second position sensing unit 140 may be anAscension Flock of Birds, Nest of Birds, driveBAY, medSAFE, trakSTAR,miniBIRD, MotionSTAR, or pciBIRD, and tracking units 145 may be magnetictracking coils. In some embodiments, the second position sensing unit140 may be an Aurora® Electromagnetic Measurement System using sensorcoils for tracking units 145. In some embodiments, the first positionsensing unit 110 may also be an optical 3D tracking system usingfiducials as tracking units 145. Such optical 3D tracking systems mayinclude the NDI Polaris Spectra, Vicra, Certus, PhaseSpace IMPULSE,Vicon MX, InterSense IS-900, NaturalPoint OptiTrack, Polhemus FastTrak,IsoTrak, or Claron MicronTracker2.

Tracking unit 145 as used herein is a broad term and includes withoutlimitation all types of magnetic coils or other magnetic field sensingdevices for use with magnetic trackers, fiducials or other opticallydetectable markers for use with optical trackers, such as thosediscussed above and below. Tracking units 145 could also include opticalposition sensing devices such as the HiBall tracking system and thefirst and second position sensing units 110 and 140 may be HiBalltracking systems. Tracking units 145 may also include a GPS device orsignal-emitting device that would allow for tracking of the positionand, optionally, orientation of the tracking unit. In some embodiments,a signal-emitting device might include a radio-frequency identifier(RFID). In such embodiments, the first and/or second position sensingunit 110 and 140 may take in the GPS coordinates of the tracking units145 or may, for example, triangulate the radio frequency signal beingemitted by the RFID associated with tracking units 145.

The first position sensing unit 110 may be used to track the position ofmovable imaging unit 155. Tracking the position of movable imaging unit155 allows for the determination of the relative emplacement, whereemplacement may refer to position and orientation or merely position, ofimaging data received using the movable imaging unit 155 and imagingunit 150 with that data being sent to image guidance unit 130. Forexample, image guidance unit 130 may contain CT data which is beingupdated and deformed based on the relative emplacements of trackingunits 145 as received by the second position sensing unit 140. In suchan example embodiment, the image guidance unit 130 may take in theemplacements, such as positions and orientations, of the tracking units145 and from that determine an updated model for CT data stored inimaging guidance unit 130. Further, imaging guidance unit 130 mayproduce images based on the current ultrasound imaging data coming fromimaging unit 150 and also based on an updated model determined based onthe emplacements of tracking units 145. The images produced 125 made bepresented on display unit 120. An example image 125 is shown in FIG. 8.

In some embodiments, a movable imaging unit 155 may not be connecteddirectly to an imagining unit 150, but may instead be connected toimaging guidance unit 130. The movable imaging unit 155 may be usefulfor allowing a user to indicate what portions of a first set of imagingdata should be displayed. For example, if the movable imaging unit 155may be an ultrasound transducer or a tracked operative needle, forexample, and may be used by a user to indicate what portions of apre-operative CT scan to show on a display unit 120 as image 125.Further, in some embodiments, there could be a third set ofpre-operative imaging data that could be displayed with the first set ofimaging data. Yet further, in some embodiments, each of the first andthird sets of imaging data could be deformed based on updated positionsof the tracking units 145 and the updated, deformed versions of the twosets of imaging data could be shown together or otherwise provide imageguidance images 125 for presentation on display 120.

First position sensing unit 110 may be an optical tracker, a magnetictracker, or any other appropriate type of position sensing device. Forexample, in various embodiments, first position sensing unit 110 may bean Ascension Flock of Birds, Nest of Birds, driveBAY, medSAFE, trakSTAR,miniBIRD, MotionSTAR, or pciBIRD. In some embodiments, the firstposition sensing unit may be an Aurora® Electromagnetic MeasurementSystem using sensor coils. In some embodiments, the first positionsensing unit 110 may also be an optical 3D tracking system such as theNDI Polaris Spectra, Vicra, Certus, PhaseSpace IMPULSE, Vicon MX,InterSense IS-900, NaturalPoint OptiTrack, Polhemus FastTrak, IsoTrak,or Claron MicronTracker2. The first position sensing unit 110 senses theposition of movable imaging unit 155. If first position sensing unit 110is an optical tracker, then movable imaging unit 155 may have fiducialsplaced thereon to make visual position and/or orientation detectionpossible. If first position sensing unit 110 is a magnetic tracker, thenmovable imaging unit 155 they have placed thereon magnetic trackingunits.

In some embodiments, the display unit 120 displays 3D images to a user.This can be accomplished by a stereoscopic display, a lenticularauto-stereoscopic display, or any other appropriate type of display. Insome embodiments, a user may wear a head mounted display in order toreceive 3D images from the image guidance unit 130. In such embodiments,a separate display, such as the pictured display unit 120, may beomitted.

In some undepicted embodiments, there is no first position sensing unit110 and the emplacements of both the movable imaging unit 155 andtracking units 145 are determined using the second position sensing unit140. Similarly, in some embodiments, the first position sensing unit 110may track the emplacements of both the movable imaging unit 155 andtracking units 145 and the second position sensing unit 140 may not bepresent.

II. Anatomical Sites and Deformation

FIG. 2 illustrates an example of an anatomical site 210 with trackingunits 145A, 145B, and 145C. An anatomical site 210 can be anywherewithin or on the body 160, human or otherwise. In some embodiments, asshown in FIG. 4, tracking units 145A, 145B, and 145C may be implantableneedles containing a magnetic tracking coil. If the tracking units 145A,145B, and 145C are implantable, then the tracking units may be placed inor near an anatomical site 210. In some embodiments, tracking units145A, 145B, and 145C may be placed on the surface of an anatomical site210. In yet other embodiments, tracking units 145A, 145B, and 145C ofknown dimensions may be placed partially inside and partially externalto an anatomical site 210. In such embodiments, a portion of thetracking unit 145A, 145B, and 145C may be in or near the anatomical site210 while another portion may be external to the body or the anatomicalsite and allow tracking external to the body. This embodiment is useful,for example, if it is desired that the second position sensing unit 140be an optical tracker or if there are other reasons, such as the size ofa magnetic tracking coil, for not implanting that portion of thetracking units 145A, 145B, and 145C.

FIG. 3 illustrates the deformation of an anatomical site 210 to 310 withtracking units 145A, 145B, and 145C. In FIG. 3, three tracking units145A, 145B, and 145C have been placed near an anatomical site 210. Theletters on the vertices of anatomical site 210 illustrate thatanatomical site 210 may be deformed into deformed anatomical site 310.The letters illustrate which vertices in anatomical site 210 correspondto which vertices in deformed anatomical site 310. The deformation ofanatomical site 210 into anatomical site 310 can be due to patientmovement, breathing, pressure, force, or any other effect that maydeform deformable tissue comprising and/or surrounding anatomical sites210 and 310. FIG. 3 illustrates that the tracking units 145A, 145B, and145C move from locations on anatomical sites 210 to correspondinglocations in deformed anatomical site 310.

In some embodiments, the first set of imaging data, such as a CT scan,MRI, open-magnet MRI, fluoroscopy, PET scan, 3D ultrasound, or any othertype of imaging data may be received in a format that is usable toperform the deformation techniques described herein. In otherembodiments, when the first set of imaging data is received, a 3D modelof that data may be produced. There are many known techniques forproducing 3D models such as finite element models, volumetric models, orpolygonal models. These include manual, human-driven techniques, such astracing the boundaries of organs and tumors (also known as contouring),and automatic techniques such as iso-surface extraction (marching cubes,watershed), or hybrid techniques such as m-rep based segmentation. Whenthe updated emplacement, such as position and orientation, or mereposition, of the tracking units 145A, 145B, and 145C is determined, amodel of the anatomical sites 210 can be updated to estimate thedeformed anatomical site 310. This updating may be accomplished usingknown techniques for the various underlying models. For example, in somecomputer graphics hardware systems, one can use 3D textures. Eachtracking unit can be associated with a texture location within the 3Dtexture, where the 3D texture comprises the first set of imaging data.Once updated positions of the tracking units 145A, 145B, and 145C areknown, then the 3D texture can be updated using linear interpolation.Image guidance unit 130 may contain general hardware, such as a CPU, orspecialized hardware, such as a graphics card, that is capable ofperforming linear interpolation on 3D textures. Relatedly, if aparticular slice of the second set of imaging data is desired fordisplay as part of image guidance data, then the particular slice fordisplay may be determined based on the corresponding slice of theupdated 3D texture. See, e.g., Yinghui, C., Jing, W., and Xiaohui, L.2006, Real-time deformation using modal analysis on graphics hardware,in Proceedings of the 4th international Conference on Computer Graphicsand interactive Techniques in Australasia and Southeast Asia (KualaLumpur, Malaysia, Nov. 29-Dec. 2, 2006). GRAPHITE '06. ACM, New York,N.Y., 173-176. Another example embodiment of linear deformation isdiscussed below. Other embodiments, bi-cubic or higher-orderinterpolation may also be used. Additionally, in some embodiments, thetracking units 145A, 145B, and 145C provide position data and notorientation data. Deformation of the 3D model can be accomplished basedon the position of the tracking units 145A, 145B, and 145C. In someembodiments, the tracking units 145A, 145B, and 145C will provide bothposition and orientation data. The additional information on orientationcan be used to provide a different kind of deformation of the model.

In some embodiments, once a new 3D model for deformed anatomical site310 is determined, the updated model can be used, for example, by imageguidance unit 130 of FIG. 1 in order to produce image guidance data thatis based on a combination of imaging data from imaging unit 150 andmovable imaging unit 155. This image guidance data may be displayed asimaging data 125 on display unit 120.

FIG. 4 depicts an embodiment of a tracking unit 145. In someembodiments, the tracking unit 145 comprises a shaft 410, a magneticcoil 430, and a cable 420. The shaft 410 may be a hollow needle or anyimplantable unit. The shaft 410 may be hollow in order to accommodateinsertion of the cable 420 and magnetic coil 430, or may be solid, inwhich case magnetic coil 430 and cable 420 must be built into the shaft410 or the shaft 410 must be constructed around the magnetic coil 430and cable 420. As noted above, in some embodiments a tracking unit 145may include an optical device, such as a fiducial (not pictured), inorder to allow proper tracking. In yet other embodiments, a trackingunit 145 may include an implantable portion in addition to and separatefrom the tracking portion (not pictured).

FIG. 5 illustrates another example of an anatomical site 210 withindeformable tissue 510 with implanted tracking units 145A, 145B, 145C,and 145D. FIG. 5 illustrates that the deformable tissue 510 deforms intodeformable tissue 520. Similar to the embodiments shown in FIG. 3,anatomical site 210 is deformed into deformed anatomical site 310.Tracking units 145A, 145B, 145C, and 145D are shown implanted nearanatomical site 210. The tracking units 145A, 145B, 145C, and 145Dremain near the anatomical site after deformable tissue 510 has beendeformed into deformed deformable tissue 520, and anatomical site 210has deformed into deformed anatomical site 310. As was the case in theembodiment shown in FIG. 3, a 3D model of anatomical site 210 can bedeformed and updated based on the relative emplacements of the trackingunits 145A, 145B, 145C, and 145D at the time before and afterdeformation.

III. Process for Providing Guidance Data

FIG. 6 depicts one of many possible example processes 600 for providingguidance data based on updated deformable tracking information. In someembodiments, all or portions of process 600 may be performed by imageguidance system 130 or by any other appropriate unit or module. In step610, a first set of imaging data from anatomical site 210 and trackingunits 145 is obtained. This first set of imaging data may be a CT scan,MRI, open-magnet MRI, fluoroscopy, PET scan, 3D ultrasound, or any otherimaging data. Obtaining the relative emplacements of the tracking units145 when taking the data for the anatomical site 210 provides theability to determine how the relative emplacements of the tracking units145 have changed, from the time that the first set of imaging data istaken, and until any time later at which the emplacements of thetracking units are known. Generally, the tracking units 145 will bevisible in the first set of imaging data, but this is not necessary. Thetracking units 145 must simply be close enough to an anatomical site ofinterest to provide information on deformation of the anatomical site.

In some embodiments, once a first set of imaging data of the anatomicalsite and tracking units 145 is obtained, a 3D model of the first set ofimaging data is produced. In other embodiments, a 3D model of the firstset of imaging data is produced at a later time or is not produced atall, and deformation of the first set of imaging data is accomplishedwithout using a 3D model. The production of a 3D model from the firstset of imaging data is discussed above.

At some time after the first of imaging data is obtained in step 610, asecond set of imaging data of the anatomical site is obtained in step620. The second set of imaging data may, like the first set of imagingdata, be any of a variety of types of imaging data. For example, asecond set of imaging data may be 2D ultrasound, 3D ultrasound,fluoroscopy, or any other type of imaging data. The tracking units 145may, but need not, be visible in the second set of imaging data. Forexample, if the second set of imaging data is 2D ultrasound, then theultrasound image obtained may include a plane or slice of the anatomicalsite 210, but tracking units 145 need not be visible in that particularslice or plane.

In step 630, a deformed version of the first set of imaging data isdetermined. In some embodiments, updated emplacements of the trackingunits 145, at the time the second set of imaging data is obtained, areused to determine an updated or deformed model of the first set ofimaging data. This is discussed above. Once the updated or deformedversion of the first set of imaging data and the recently obtainedsecond set of imaging data are both available, the relative emplacementsof those two sets of imaging data are determined. This may beaccomplished based on both the emplacements of the tracking units 145,in order to determine the emplacement of the deformed version of thefirst set of imaging data, and the emplacement of the second set ofimaging data. The emplacements of the second set of imaging data may bedetermined based on, for example, the location of a movable imaging unit155, as depicted in FIG. 1. As another example embodiment, if a movableimaging unit 155 is a 2D ultrasound wand, then tracking the location ofa movable imaging unit 155 allows determination of the position andorientation of the second set of imaging data. As noted above, themovable imaging unit 155 may be tracked using the first position sensingunit 110. In some embodiments, the determination of relativeemplacements of the two imaging data sets may take the form of a 3Dtransformation or other mathematical relationship.

Once the relative positions of the second set of imaging data and thedeformed version of the first set of imaging data are known, the imageguidance data can be determined and displayed in step 650. In someembodiments, the image guidance data shows features within the deformedversion of the first set of imaging data in combination with a secondset of imaging data, such as that depicted in FIG. 8. In otherembodiments, the image guidance data may be an overlay or a combinationof the second set of imaging data and of the deformed version of thefirst set of imaging data. Other examples of image guidance data thatmay be displayed in step 650 are those depicted in FIGS. 7A-7D,discussed below.

In some embodiments, each time that more imaging data is received fromthe data source that provided the second set of imaging data, process600 may repeat starting at step 620. Process 600 may also be restartedfrom step 610, especially in scenarios, such as resection at theanatomical site, where warping of the first set of imaging data is nolonger possible. In such a case, the first set of imaging data may bere-obtained in step 610.

As noted above, in some embodiments, the deformation is accomplished bylinear deformation. Linear deformation may be accomplished in number ofways, including using graphics hardware. As one example embodiment oflinear deformation, consider an original volume image I (as scanned, forexample, by the CT scanner at time t) as an anatomical site of interest.At time t, there are n tracking units implanted in the tissue near andaround the anatomical site. The tracking units' positions are pt₁ . . .pt_(n). For each tracking unit, we compute the 3d texture coordinate,tc, that indicates the position of the tracking sensor, in image I'scoordinate system. 3d texture coordinates may lie in the range u=[0 . .. 1], v=[0 . . . 1], w=[0 . . . 1]. The eight corners of the image I inthe 3d texture's coordinate system may be (0,0,0), (0,0,1), (0,1,0),(0,1,1), (1,1,0), (1,1,1). The points may then be stored in a table-likedata structure as follows:

Points table: point #1, pt₁.x, pt₁.y, pt₁.z, tc₁.u, tc₁.v, tc₁.w, point#2, pt₂.x, pt₂.y, pt₂.z, tc₂.u, tc₂.v, tc₂.w, point #3, pt₃.x, pt₃.y,pt₃.z, tc₃.u, tc₃.v, tc₃.w, . . . point #n, pt_(n).x, pt_(n).y,pt_(n).z, tc_(n).u, tc_(n).v, tc_(n).w,where pt_(k).y and tc_(k).u refer, for example, to point pt_(k)'s ycoordinate and tc_(k)'s u coordinate, respectively.

The volume may then be tessellated into tetrahedra. The corner point ofeach tetrahedron k may be one of n tracking units, at position pt_(k).The edges of the tessellation may be stored in a data structure asfollows:

Edges table: edge #1, point a₁, point b₁, edge #2, point a₂, point b₂, .. . edge #m, point a_(n), point b_(n),where each edge connects two points, each of which is a reference to apoint #. For example, point a₁ may refer to point #1 and point b₁ mayrefer to point #4.

Tetrahedra table: tetrahedron #1, edge a₁, edge b₁, edge c₁, edge d₁,edge e₁, edge f₁, tetrahedron #2, edge a₂, edge b₂, edge c₂, edge d₂,edge e₂, edge f₂, . . . ,where edge a₁ and a₂ may refer, for example, to edges #5 and edge #2,respectively.

Some of the image I may lie outside of the convex hull of the points pt₁. . . pt_(n). These portions of image I may be ignored or otheralgorithms may be used to determine their distortion.

At time j, where j>t, the tissue may have changed shape, and image I(which represents the anatomical site at time t) may no longer representthe. The positions of the tracking sensors at time j, are pj₁ . . .pj_(n). One may consider a new image J that is a linearly warped copy ofimage I. One may not need to compute image J, however. Instead, at timej, an updated, deformed image for a 2d cross-sectional plane throughimage J may be determined as follows:

-   -   Update the points table, and replace each pt_(k) with pj_(k)        where k={1 . . . n}, thereby updating the position of each        tracking unit, at time j, as supplied, for example, by position        sensing system 140.    -   Iterate through the tetrahedra table. For each tetrahedron,        iterate through each of its six edges. For each such edge,        compute its intersection with the cross-sectional plane. For        those edges that do intersect with the plane, we compute the        intersection point P. We then compute the texture coordinate for        P, by linearly interpolating between the texture coordinates at        its endpoints (those texture coordinates are stored in the        points table).        -   For each tetrahedron, there will be 0, 1, 2, 3, or 4 edges            that intersect with the plane (resulting in 0, 1, 2, 3, or 4            intersections points P, and their corresponding texture            coordinates).    -   When there are 0, 1, or 2 intersections, render nothing        associated with the tetrahedron.    -   When there are 3 or 4 intersections of the cross-sectional        plane, render a textured polygon using traditional graphics        hardware commands (e.g. OpenGL or DirectX). The texture        coordinates index into the image I, but because the point P will        have moved from the original location, the result will be a        portion of image J.    -   Repeat this for each tetrahedron. After all tetrahedra have been        processed, the graphics hardware will have rendered the        intersection of the warped image J, with the chosen        cross-sectional plane.

In some embodiments, if the second set of imaging data is a planarfluoroscopic image, then the deformed version of the first set ofimaging data may be projected onto the plane of the planar fluoroscopyand the two images may be combined in order to produce the imageguidance data.

In some embodiments, the image guidance data determined in step 650 fromthe deformed version of the first set of imaging data could be used toapproximate another imaging modality. For example, the first set ofimaging data may be a CT scan and a user may wish to have anapproximation of a biplane fluoroscopy performed without exposing apatient to the harmful radiation associated with such a fluoroscopy. Thedeformed version of the first set of imaging data may be projected ontowhat would be the two planes of the biplane fluoroscopy. This wouldapproximate the biplane fluoroscopy using the updated tracking unitinformation without exposing the patient to the radiation associatedwith the biplane fluoroscopy. Further, this approximation could beupdated at a rate that exceeds that of conventional biplane fluoroscopyas the tracking units move with the surrounding tissues to which theyare affixed, without harming the patient or disturbing the ongoingoperation.

IV. Creating Features in Imaging Data

FIGS. 7A-7D depict marking and viewing features in imaging data.Generally FIGS. 7A-7D show the manual creation of features within afirst set of imaging data and updating the placement of the featurebased on the deformed model of the first set of imaging data. FIG. 7Ashows a feature selection unit 710 being used to highlight, in aparticular plane or visual slice of the first set of imaging data 720, afeature 730 within the first of imaging data. In FIG. 7B, the userpoints to the feature 730 and selects the feature in order to signifyselected feature 740. When the first set of imaging data 720 is beingviewed later, the selected feature 740 will still appear in its originalposition, as depicted in FIG. 7C.

FIG. 7D illustrates that if a plane of the first set of imaging data 720is displayed, and the selected feature 740 may still be displayed evenif it is not within the visual slice or plane or display data of thefirst set of imaging data 720. The displacement of the feature 740 fromthe visual slice of the first set of imaging data 720 may be shown witha displacement marker 750. In some embodiments, the displacement of theselected feature 740 from the visual slice of the first set of imagingdata 720 may be shown with other visual techniques, such as visual depthon a 3D display, shadowing, foreshortening, or any other knowntechnique. In FIGS. 7C and 7D, even if the first set of imaging data isdeformed based on updated emplacements of the tracking units 145 (notpictured), the selected feature 740 may be shown in a new location basedon the deformation of the first set of imaging data 720. Marking thesefeatures may be useful so that a user or surgeon could identify andlater find points of interest, such as locations of tumors or lesions.

V. Image Guidance Data

FIG. 8 depicts an embodiment of image guidance data in which a first setof imaging data is presented with a second set of imaging data. In someembodiments, the second set of imaging data is displayed inapproximately the form that is received as described above. The first ofimaging data may be deformed based on the updated emplacements oftracking units 145. The updated or deformed model corresponding to thefirst set of imaging data may be used to provide image guidance, such asthe location of an important feature of the anatomical site. Forexample, in the ultrasound image of the liver depicted in FIG. 8, theultrasound image 810 (the second set of imaging data) is augmented witha feature 820 from the first set of imaging data, such as a CT scan.This feature may be the location of a tumor, necrosed tissue, or anyother relevant feature. It may have been detected or selected by a user,as illustrated in FIGS. 7A-7D. Upon deformation of the underlyingtissue, the model for the first set of imaging data may be deformed andtherefore the feature 820 shown from the second set of imaging datawould also be updated and deformed. Over time, as new second sets ofimaging data 810 were being generated, the movement and updatedemplacements of the tracking units 145 in an anatomical site would causethe deformation and movement of the feature of the first set of imagingdata 820. The updated emplacement of the feature 820 would continue tobe shown with the newly received second set of imaging data 810. In someembodiments, the deformation of the first set of imaging dataapproximates the deformation of the underlying tissue. Therefore, theapproximate placement of the feature 820 would approximate the locationof the underlying anatomical feature within the second set of imagingdata 810.

The processes, computer readable medium, and systems described hereinmay be performed on various types of hardware, such as computer systems.Computer systems may include a bus or other communication mechanism forcommunicating information, and a processor coupled with the bus forprocessing information. A computer system may have a main memory, suchas a random access memory or other dynamic storage device, coupled tothe bus. The main memory may be used to store instructions and temporaryvariables. The computer system may also include a read-only memory orother static storage device coupled to the bus for storing staticinformation and instructions. The computer system may also be coupled toa display, such as a CRT or LCD monitor. Input devices may also becoupled to the computer system. These input devices may include a mouse,a trackball, or cursor direction keys. Computer systems described hereinmay include the image guidance unit 130, first and second positionsensing units 110 and 140, and imaging unit 150. Each computer systemmay be implemented using one or more physical computers or computersystems or portions thereof. The instructions executed by the computersystem may also be read in from a computer-readable medium. Thecomputer-readable medium may be a CD, DVD, optical or magnetic disk,laser disc, carrier wave, or any other medium that is readable by thecomputer system. In some embodiments, hardwired circuitry may be used inplace of or in combination with software instructions executed by theprocessor.

As will be apparent, the features and attributes of the specificembodiments disclosed above may be combined in different ways to formadditional embodiments, all of which fall within the scope of thepresent disclosure.

Conditional language used herein, such as, among others, “can,” “could,”“might,” “may,” “e.g.,” and the like, unless specifically statedotherwise, or otherwise understood within the context as used, isgenerally intended to convey that certain embodiments include, whileother embodiments do not include, certain features, elements and/orstates. Thus, such conditional language is not generally intended toimply that features, elements and/or states are in any way required forone or more embodiments or that one or more embodiments necessarilyinclude logic for deciding, with or without author input or prompting,whether these features, elements and/or states are included or are to beperformed in any particular embodiment.

Any process descriptions, elements, or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or steps in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved, as would be understood by those skilled in the art.

All of the methods and processes described above may be embodied in, andfully automated via, software code modules executed by one or moregeneral purpose computers or processors, such as those computer systemsdescribed above. The code modules may be stored in any type ofcomputer-readable medium or other computer storage device. Some or allof the methods may alternatively be embodied in specialized computerhardware.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

1. A method of presenting imaging data related to an anatomical site,comprising: obtaining a first set of imaging data related to ananatomical site at a first time; obtaining information on one or moretracking units at the anatomical site at the first time; obtainingtracking information for a movable imaging device controlled by a userat a second time, after the first time; obtaining information on the oneor more tracking units at the anatomical site at the second time;determining desired emplacement information for an image of the firstset of imaging data based on the tracking information; determining adeformed version of the first set of imaging data based on a firstlocation of the one or more tracking units at the first time and asecond location of the one or more tracking units at the second timethat is different from the first location; and determining imageguidance data for display based on the first set of imaging data, thedeformed version of the first set of imaging data and the desiredemplacement information.
 2. The method of claim 1, wherein the methodfurther comprises: obtaining at a third time, before the second time, asecond set of imaging data; and wherein: determining image guidance datafor display comprises determining image guidance data for display basedon the first set of imaging data, the deformed version of the first setof imaging the second set of imaging data, and the desired emplacementinformation.
 3. The method of claim 1, wherein determining imageguidance data for display comprises approximating a bi-planefluoroscopy.
 4. The method of claim 1, wherein obtaining a first set ofimaging data related to the anatomical site comprises obtaining anarrangement of tracking units at the anatomical site.
 5. The method ofclaim 1, further comprising receiving information about a feature withinthe first set of imaging data, wherein determining image guidance datafor display comprises displaying the feature within the first data set.6. The method of claim 1, wherein the movable imaging device is amovable ultrasound device.
 7. The method of claim 1, wherein determininga deformed version of the first set of imaging data comprises performinga linear interpolation on a model of the first set of imaging data. 8.The method of claim 1, wherein determining a deformed version of thefirst set of imaging data comprises performing a bi-cubic interpolationon a model of the first set of imaging data.
 9. The method of claim 1,wherein determining the deformed version of the first set of imagingdata comprises performing a deformation on a volumetric model of thefirst set of imaging data based on the arrangements of the trackingunits at the first time and at the second time.
 10. The method of claim2, wherein the second set of imaging data is associated with the movableimaging device and the emplacement of the second set of imaging data isdetermined using the emplacement of the movable imaging device.
 11. Themethod of claim 2, wherein the second set of imaging data is associatedwith movable imaging device and the emplacement of the second set ofimaging data is determined using the emplacement of a second movabledevice, wherein the movable imaging device is distinct from the secondmovable device.
 12. The method of claim 4, wherein obtaining anarrangement of the tracking units comprises determining the location andorientation of the tracking units.
 13. The method of claim 4, whereinobtaining the first set of imaging data comprises obtaining a volumetricmodel of obtained imaging data including the emplacement of the trackingunits, wherein the method further comprises determining a deformedversion of the first set of imaging data by performing a deformation ofthe volumetric model.
 14. The method of claim 4, wherein automaticallyobtaining the first set of imaging data comprises obtaining a finiteelement model of obtained imaging data including the emplacement of thetracking units; wherein the method further comprises determining adeformed version of the first set of imaging data by performing adeformation of the finite element model.
 15. The method of claim 11,wherein the movable imaging device is a movable 3D ultrasound device andthe second movable device is a display indication device.
 16. A systemfor presenting imaging data related to an anatomical site, comprising:an image guidance system in communication with one or more trackingunits at an anatomical site, the image guidance system configured to:obtain a first set of imaging data related to the anatomical site at afirst time; obtain location information of the one or more trackingunits at the first time; obtain tracking information for a movableimaging device controlled by a user at a second time, after the firsttime; obtain location information of the one or more tracking units atthe second time; determine desired emplacement information for an imageof the first set of imaging data based on the tracking information; anddetermine a deformed version of the first set of imaging data based on afirst location of the one or more tracking units at the first time and asecond location of the one or more tracking units at the second timethat is different from the first location; and determine image guidancedata for display based on the first set of imaging data, the deformedversion of the first set of the first set of imaging data, and thedesired emplacement information.
 17. A computer-readable medium thatexecutes the steps for presenting imaging data related to an anatomicalsite, comprising: obtaining a first set of imaging data related to ananatomical site at a first time; obtaining information on tracking unitsat the anatomical site at the first; obtaining tracking information fora movable imaging device controlled by a user at a second time, afterthe first time; obtaining information on the tracking units at theanatomical site at the second time; determining desired emplacementinformation for an image of the first set of imaging data based on thetracking information; determining a deformed version of the first set ofimaging data based on a first location of the tracking units at thefirst time and a second location of the tracking units at the secondtime that is different from the first location; and determining imageguidance data for display based on the first set of imaging data, thedeformed version of the first set of imaging data, and the desiredemplacement information.